Papers by Steven C.H. Hoi
Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understanding (2022.emnlp-main)
Copied to clipboard
| Challenge: | Prompt Tuning has been successful as a parameter-efficient method of conditioning large-scale pre-trained language models to perform downstream tasks. |
| Approach: | They propose to use a vector-quantized input-contextualized prompt as an extension to the soft prompt tuning framework to learn contextualization of soft prompt tokens. |
| Outcome: | The proposed prompt outperforms soft prompt tuning by an average margin of 1.19% on various language understanding tasks like SuperGLUE, QA, Relation classification, NER and NLI. |
LAVIS: A One-stop Library for Language-Vision Intelligence (2023.acl-demo)
Copied to clipboard
| Challenge: | a new open-source library for language-vision research and applications is available for free. |
| Approach: | They introduce LAVIS, an open-source deep learning library for LAnguage-VISion research and applications. |
| Outcome: | The proposed library is open-source and highly extensible and configurable. |
Detect-Localize-Repair: A Unified Framework for Learning to Debug with CodeT5 (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Automated software debugging is crucial for improving productivity of software developers . many neural-based techniques focus only on one or the other, ignoring mutual benefits . |
| Approach: | They propose a framework to adapt a pretrained programming language model to automate debugging . they propose three objectives: bug detection, bug localization, program repair . |
| Outcome: | The proposed framework outperforms baselines from both NLP and software engineering domains on two new datasets. |
VD-BERT: A Unified Vision and Dialog Transformer with BERT (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work focused on attention mechanisms to model complex interactions in visual dialog . a new framework for visual dialog is based on pretrained BERT language models . |
| Approach: | They propose a framework for a vision-dialog Transformer that leverages pretrained BERT language models for Visual Dialog tasks. |
| Outcome: | The proposed framework achieves the top position on the visual dialog leaderboard without pretraining on external vision-language data. |
UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to training dialogue agents separately are not optimized for multi-domain task-oriented dialogues. |
| Approach: | They propose a unified neural architecture for end-to-end conversational systems in multi-domain task-oriented dialogues that jointly trains a bi-level state tracker and a joint dialogue act and response generator. |
| Outcome: | The proposed system outperforms existing systems on the MultiWOZ2.1 benchmark in dialogue state tracking, context-to-text, and end-to end settings. |
Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading (2020.acl-main)
Copied to clipboard
Yifan Gao, Chien-Sheng Wu, Shafiq Joty, Caiming Xiong, Richard Socher, Irwin King, Michael Lyu, Steven C.H. Hoi
| Challenge: | Existing approaches to answer user questions are limited in their decision making due to struggles in extracting question-related rules and reasoning about them. |
| Approach: | They propose a conversational machine reading framework that uses a Explicit Memory Tracker to track whether conditions in the rule text have already been satisfied to make a decision. |
| Outcome: | The proposed framework achieves state-of-the-art on the ShARC benchmark and is more interpretable by visualizing the entailment-oriented reasoning process as the conversation flows. |
Improving Limited Labeled Dialogue State Tracking with Self-Supervision (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing dialogue state tracking models require plenty of labeled data, but collecting labels is expensive. |
| Approach: | They propose to use only 1% labeled data to train dialogue state tracking models . they encourage a model to have consistent latent distributions given a perturbed input . |
| Outcome: | The proposed self-supervised signals improve goal accuracy by 8.95% when only 1% labeled data is used on the MultiWOZ dataset. |
Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading (2020.emnlp-main)
Copied to clipboard
Yifan Gao, Chien-Sheng Wu, Jingjing Li, Shafiq Joty, Steven C.H. Hoi, Caiming Xiong, Irwin King, Michael Lyu
| Challenge: | Document interpretation and dialog understanding are the two major challenges for conversational machine reading. |
| Approach: | They propose a discourse-aware entailment reasoning network to strengthen the connection and enhance the understanding of document and dialog. |
| Outcome: | The proposed model improves document interpretation and dialog understanding on the ShARC benchmark. |
TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models with self-attention encoder architectures are less useful in practice. |
| Approach: | They propose to use user and system tokens to model dialogue behavior during pre-training . they propose a contrastive objective function to simulate the response selection task . |
| Outcome: | The proposed model outperforms baseline models on four downstream tasks . it also has a few-shot ability that can mitigate the data scarcity problem . |
Video-Grounded Dialogues with Pretrained Generation Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Pre-trained language models have shown success in improving downstream NLP tasks . pre-tuned models capture textual dependencies in text data of rich semantics . |
| Approach: | They propose a framework for improving video-grounded dialogue by extending GPT-2 models . they propose to combine visual and textual representation into a structured sequence . |
| Outcome: | The proposed framework improves audio-visual scene-aware dialogues benchmark on AVSD . it is based on a large pre-trained GPT-2 network and can generate natural responses . |
Learning Label Modular Prompts for Text Classification in the Wild (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in parameter efficient tuning of pretrained language models have limited performance. |
| Approach: | They propose a label-modular prompt tuning framework for text classification tasks that emulates the transient nature of real-world. |
| Outcome: | The proposed framework outperforms baselines in two formidable settings and shows strong generalisation ability. |
BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialogues (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to video-grounded dialogues focus on superficial temporal-level visual cues, but neglect more fine-grained spatial signals from videos. |
| Approach: | They propose a vision-language neural framework for high-resolution queries in videos based on textual cues that exploits both spatial and temporal-level information. |
| Outcome: | The proposed approach outperforms previous approaches on the TGIF-QA benchmark and significantly outperformed previous approaches. |
Photon: A Robust Cross-Domain Text-to-SQL System (2020.acl-demos)
Copied to clipboard
Jichuan Zeng, Xi Victoria Lin, Steven C.H. Hoi, Richard Socher, Caiming Xiong, Michael Lyu, Irwin King
| Challenge: | Existing natural language interfaces to databases are ambiguous or untranslatable . we present a robust, modular cross-domain text-to-SQL system . |
| Approach: | They propose a system that flags natural language input to which a SQL mapping cannot be immediately determined. |
| Outcome: | The proposed system can flag natural language input to which a SQL mapping cannot be determined. |
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained models for Natural Languages (NL) like BERT and GPT have been shown to transfer well to Programming Languages. |
| Approach: | They propose a unified pre-trained encoder-decoder Transformer model that leverages the code semantics conveyed from the developer-assigned identifiers. |
| Outcome: | The proposed model outperforms existing models on understanding and generation tasks and can capture semantic information from code. |
Response Selection for Multi-Party Conversations with Dynamic Topic Tracking (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing response selection methods focus on a two-party single-conversation scenario. |
| Approach: | They propose a multi-task learning framework that frames response selection as a dynamic topic tracking task to match the topic between the response and relevant conversation context. |
| Outcome: | The proposed framework outperforms existing methods on an Ubuntu IRC dataset in response selection and topic disentanglement tasks. |
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches require substantial adaptation of pretrained language models for vision-language reasoning tasks. |
| Approach: | They propose to use natural language and network interpretation as an intermediate representation that glues pretrained models together. |
| Outcome: | The proposed framework outperforms the Flamingo model on VQAv2 and GQA by 8.5%. |